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Significant  enhancements  to  the  NAVO  MSRC 
computing  capability  are  on  the  way  this  spring 
and  summer,  thanks  to  the  Technology 
Insertion  for  FY04  (TI-04)  upgrades  that  were 
recently  approved  by  our  sponsors. 

These  enhancements  include  substantial 
upgrades  to  the  MSRC  High  Performance 
Computing  (HPC)  systems  configuration,  the 
Remote  Storage  Facility  (RSF),  and 
the  internal  MSRC  networking  capability. 

When  the  TI-04  upgrades  are  complete,  they 
will  provide  almost  30  teraflops  of  aggregate 
peak  computing  capability  with 
commensurately  balanced  storage  and 
networking  capabilities — including  the 
installation  of  one  of  the  world's  most  capable 
HPC  systems,  a  2944-processor  IBM 
POWER4-I-  system.  This  enormous 
computational  capability  will  continue  to 
enable  unparalleled  advances  in  the  DoD 
science  and  technology  areas  served  by  the 
HPC  Modernization  Program  (HPCMP). 

As  I've  mentioned  in  past  issues  of  The 
Navigator,  we  recognize  that  it  is  critically 
important  for  us  to  redouble  our  efforts  in 
assessing  and  implementing  common  user 
environments,  practices,  and  tools  within  and 
across  the  centers.  Your  individual  and 


collective  user  feedback  through  the  User 
Advocacy  Group  makes  it  clear  that  you 
consider  this  to  be  one  of  your  highest  priorities 
for  us. 

In  response,  the  HPCMP  Office  and  the  major 
centers  within  the  program  have  formalized 
and  chartered  a  Corporate  Board  of  Directors 
(CBOD),  tasked  to  develop  policies  and 


NAVO  MSRC 

System  Upgrades 
Support  Users 

strategies  in  this  regard — and  to  oversee  the 
efforts  of  the  many  technical  design, 
implementation,  and  maintenance  teams  who 
will  contribute  to  this  common  environment. 

My  staff  and  I  look  forward  to  seeing  you  in 
June  at  the  2004  HPCMP  Users'  Conference  in 
Williamsburg,  Virginia.  As  always,  please  take 
every  opportunity  to  let  us  know  how  we  can 
better  serve  you — ^your  feedback  is  critically 
important  to  us  and  to  the  HPCMP. 


About  the  Cover: 

Pictured  are  Navy  Joint  Strike  Fighter  (JSF)  F-35B  Jets,  future  generations  of  which  will  benefit  from  the 
research  described  in  State-to-State  Kinetics-Based  Prediction  of  Thermal  Nonequilibrium  in  Hypersonic  Flows 
(Page  9)  and  Direct  Numerical  Simulation  of  Pulsed  Jets  in  Cross-Flow  (Page  14),  as  well  as  other  projects 
underway  at  the  NAVO  MSRC. 
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Validation  of  the  1/8°  Global  Navy  Coastal 
Ocean  Model  Nowcast/Forecast  System 

Charlie  N.  Barron,  Lucy  F.  Smedstad,  Robert  C.  Rhodes,  A.  Birol  Karo,  Clark  Rowley,  Richard  A.  Allard,  and 
Harley  E.  Hurlburt,  Naval  Research  Laboratory,  Stennis  Space  Center,  MS 


One  of  the  primary  concerns  driving 
the  development  of  U.S.  Navy  global 
models  is  improved  performance  and 
nesting  support  in  shelf  and  nearshore 
regions  with  short  notice  applicability 
anywhere  on  the  globe.  The  global 
implementation  of  the  Navy  Coastal 
Ocean  Model  (NCOM)  is  a  product  of 
the  effort  at  the  Naval  Research 
Laboratory  (NRL)  to  meet  this  need.  i 

NCOM  is  the  latest  transition  of  the 
NRL  first-generation  global  system,  a 
progression  of  planned  ocean  analysis 
and  forecast  systems  delivered  for 
Navy  operations  at  the  Naval 
Oceanographic  Office  (NAVOCEANO).^ 

These  systems  include  the  Modular 
Ocean  Data  Assimilation  System 
(MODAS)3  and  the  1/16°  Navy 
Layered  Ocean  Model  (NLOM).^ 
NLOM  is  a  deep-water  ocean  model 
with  higher  horizontal  resolution,  soon 
to  be  increased  to  1/32°,  to  better 
resolve  mesoscale  ocean  dynamics. 

The  delivery  of  global  NCOM 
provides  a  medium-resolution  (1/8°  or 
14  kilometer  (km)  midlatitude)  source 
for  fully  global  upper-ocean  prediction 
and  boundary  conditions  for  higher 
resolution  coastal  models. 

Its  40  sigma-z  levels  are  concentrated 
toward  the  surface  to  maintain  a 
minimum  rest  thickness  of  1  meter  (m) 
in  the  uppermost  layer. 

In  addition,  NCOM  provides  a  capability 
to  be  the  ocean  component  for  a 
global  air-ocean  forecast  and 
embedded  Arctic  ice  models.  In 
joining  the  operational  model  suite 
at  NAVOCEANO,  NCOM  extends 
prognostic  model  coverage  to  the 
entire  global  ocean,  including  coastal, 
Arctic,  and  Antarctic  regions.  Figure  1 
illustrates  the  coverage  in  an  Atlantic 
hemispheric  view  of  assimilative 


NCOM  1998-2000  mean  Sea  Surface 
Temperature  (SST).  Together  these 
model  systems  offer  improved  Navy 
operational  global  ocean  analysis/ 
prediction  capability  in  coastal  and 
open  oceans. 

The  global  NCOM  system  consists  of 
the  numerical  model,  associated 
databases,  the  assimilation  system, 
and  data  extraction  software.  (Detailed 
descriptions  of  the  NCOM 
controlling  equations  and  numerical 
implementations  are  given  in 
Barron  et  al.^) 

NCOM  solutions  are  driven 
by  atmospheric  forcing  from 
the  Navy  Operational 
Global  Atmospheric 
Prediction  System  (NOGAPS) 
and  assimilate  temperature 
and  salinity  fields  produced 
by  MODAS  using  NLOM 
Sea  Surface  Height  (SSH). 

In  addition,  an  NRL-compiled 
global  database  of  monthly 
mean  river  discharge  has  been 
incorporated  into  NCOM,  and 
model  topography  is  based  on  a 
global,  2-minute  gridded  bottom 
topography  produced  at  NRL. 

Tides  are  not  included  in  global 
NCOM  but  are  optionally  added 
by  post-processing  using  the 
Oregon  State  University  global 
tide  constituent  databases 
Before  transitioning  global  NCOM  to 
operations,  an  evaluation  of  the 
system  was  necessary  to  quantify  the 
accuracy  of  its  products.  The 
assessments  provide  measures  of 
performance  and  model  error  that 
can  be  used  to  estimate  product 
confidence  and  improve  real-time 
assimilation.  They  also  indicate  a 
baseline  for  future  upgrades  to  the 
global  modeling  system.  A  validated 


real-time,  fully  global  ocean  modeling 
system  not  only  continually  provides 
model  nowcasts  and  forecasts 
wherever  they  are  needed,  but  also 
affords  a  level  of  confidence  in  these 
products  based  on  the  verified  level  of 
performance  in  prior  products. 

Three  interannual  1/8°  global  NCOM 
model  simulations  were  integrated  on 
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Figure  1.  Assimilative  1/8**  global  NCOM 
1998-2000  mean  SST. 


128  processors  of  the  NAVO  MSRC 
IBM  SP3  (HABU):  (1)  free-running 
(i.e.,  atmospheric  forcing  only)  1998- 
2002,  (2)  assimilative  (i.e.,  with  ocean 
data  assimilation  of  MODAS 
temperature  and  salinity  fields)  1998- 
present,  and  (3)  forecast  mode 
1998-2002  using  restarts  from  the 
assimilative  experiment. 
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Figure  2.  Assimilative  1/8**  global  NCOM  SSH  (a)  compared  with  NAVOCEANO 
25  January  -  15  March  2003  and  (b)  over  the  Arabian  Sea  on  20  March  2003. 


The  assimilative  case  corresponds  to 
the  operational  implementation 
(although  excluding  in  situ  observations 
to  allow  independent  validation), 
while  the  free-running  case  provides 
a  reference  for  assessing  stand-alone 
model  skill  and  impact  of  assimilation. 

All  NCOM  figures  in  this  article  are 
from  the  assimilative  case.  More 
information  regarding  global  NCOM 
validations  are  reported  by  Kara  et  al.'^ 
and  Barron  et  al.^ 

The  validation  test  report^  covers 
comparisons  between  unassimilated 
observations  and  a  variety  of  NCOM 


results:  temperature,  transports,  currents, 
kinetic  energy,  drifter  trajectories,  SSH, 
mixed-layer  depth,  and  boundary 
conditions.  The  validations  are  broad 
and  global  but  not  exhaustive;  more 
detailed  examinations  are  possible 
for  any  aspect  or  region.  An  example 
of  an  SSH  comparison  is  shown  in 
Figure  2a. 

Information  for  the  model  SSH  is 
extracted  at  the  location  (not  shown) 
of  a  NAVOCEANO  observing  buoy 
during  the  period  before  Operation 
Iraqi  Freedom,  and  the  two  show 
good  agreement.  Results  at  this 
specific  location  combined  with  a 


TABS  buoy  measurements 

validation  of  about  600  individual 
yearlong  SSH  time  series  around  the 
world,  showing  a  median  correlation 
of  .76,  increase  confidence  in  the 
accuracy  of  model  SSH  in  other, 
non-corroborated  locations,  such  as 
the  broader  Arabian  Sea  region  in 
Figure  2b. 

Means  of  observations  offer  another 
avenue  for  model  evaluation.  Eddy 
kinetic  energy  at  700  m  from  a  1998- 
2000  NCOM  mean  (see  Figure  3a) 
shows  similar  patterns  but  lower 
energy  than  a  mean  of  historical 
observations  taken  from  Schmitz  lo 
(see  Figure  3b). 


45^N 

3<fN 
25"  N 

(a) 


(b) 


Figure  3.  Eddy  kinetic  energy  at  700  m  from  (a)  the  1/8**  global  assimilative  global  NCOM  1998-2000  mean  and 
(b)  a  mean  of  historical  observations  taken  from  Schmitz  (1996). 


6 


SPRING  2004 


NAVO  MSRC  NAVIGATOR 


Figure  4.  Equatorial  cross  section  of  eastward  velocity  at  from  (a)  the  assimilative  1/8**  global  NCOM  1998- 

2000  mean  and  (b)  a  1991-1999  mean  taken  from  Johnson  et  al.'*'* 


Despite  different  time  spans, 
equatorial  cross  sections  of  velocity 
reflect  good  agreement  between  the 
1998-2000  NCOM  mean  seen  in 
Figure  4a  and  the  mean  calculated  by 
Johnson  et  alJi  for  1991-1999  (see 
Figure  4b),  with  the  model  yielding 
somewhat  smaller  peak  speeds  and 
showing  vertical  stretching  at  depth. 
The  resolution  and  model 
configuration  for  global  NCOM  is  an 
attempt  to  best  allocate  resources  for 
a  timely,  affordable,  and  useful  system. 
Features  and  ocean  processes  on 
horizontal  scales  of  10  K  or  less 
cannot  be  represented  by  such  a  global 


model;  these  are  instead  relegated  to 
nested  models  of  smaller,  regional 
domains  with  increasing  resolution. 

Figure  5  depicts  some  of  these 
regional  domains,  including  the  East 
Asian  Seas  (EAS)  NCOM  and  Intra- 
Americas  Seas  (IAS)  that  are  in  the 
transition  pipeline.  Global  NCOM  also 
supported  nested  relocatable  models 
for  Prestige  oil  spill  clean-up  activities 
off  the  coast  of  Spain  in  2002-2003. 

In  addition,  other  areas  and  exercises 
have  been  supported,  including 
activities  in  the  Persian  Gulf  during 
Operation  Iraqi  Freedom. 


Validation  of  the  global  NCOM  system 
has  shown  the  added  value  of  the 
model  transition.  Areas  for  future 
improvements  have  also  been 
identified.  Presently  the  authors  are 
continuing  to  improve  the  assimilation 
components  by  performing  experiments 
with  assimilation  using  the  1/32° 
NLOM,  which  is  a  planned  upgrade, 
as  soon  as  the  NAVO  MSRC 
expands  to  sufficient  operational 
computing  resources,  likely  late  2004. 
NCOM  assimilation  improvements  will 
begin  transition  on  a  similar  schedule, 
after  completion  of  the  NCOM 
operational  testing. 


Figure  5.  Snapshots  from  selected  regional  NCOM  models  that  have  been  nested  within  global  NCOM. 
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State-to-State  Kinetics-Based  Prediction  of 


Thermal  Nonequilibrium  in  Hypersonic  Flows 

Eswar  Josyula,  Computational  Sciences  Center  of  Excellence,  Air  Force  Research  Laboratory, 

Wright  Patterson  Air  Force  Base 


Sustained  hypersonic  flight  and  rapid, 
routine  access  to  space  pose  daunting 
challenges  associated  with  the  harsh 
environment  encountered  by  vehicles 
traveling  at  extremely  high  speeds. 
Figure  1  depicts  two  relevant 
examples  of  these  challenges:  the 
external  flow  around  the  X-15  aircraft 
and  internal  flow  in  a  rocket  engine. 

To  successfully  meet  these  challenges, 
testing  is  required  to  evaluate 
advanced  designs  and  technologies  to 
be  employed  on  the  next  generation 
of  hypersonic  vehicles.  However, 
there  are  many  difficulties  in 
conducting  ground  and  flight  tests. 
The  wide-ranging  spatio-temporal 
scales  that  arise  at  high  temperatures 
are  one  of  the  many  difficulties  that 
must  be  overcome  in  these  tests. 

The  cost  and  difficulty  of  conducting 
ground  and  flight  tests  accentuate  the 
need  for  complementary  simulation 
techniques  to  reduce  the  number  of 
tests.  However,  a  mathematical 
simulation  of  the  thermodynamic 
nonequilibrium  processes  of 
hypersonic  flows  is  extremely  difficult 
to  develop.  This  article  provides 
a  brief  overview  of  these 
challenges,  develops 


a  simulation  framework,  and  applies 
this  framework  to  hypersonic  flows 
around  a  vehicle  forebody  and  flows 
inside  an  expansion  nozzle. 

Challenges 

Accurate  computations  in  the  time 
scales  of  the  kinetic  processes  in  trans- 
atmospheric  flowfields  require 
knowledge  of  the  reaction  processes' 
finite  rates.  The  proper  simulation  and 
understanding  of  these 
thermodynamic  nonequilibrium 
processes  occurring  in  the  flowfields 
of  the  high  speed  systems  are  crucial 
for  prediction  of  aeroheating  and  drag 
in  external  flows  around  vehicles  and 
thrust  in  the  internal  flows  of 
propulsive  nozzles. 

The  proper  treatment  of  energy 
transfers  between  nonequilibrium 
molecular  energy  modes  and  the 
dissociation  and  ionization  processes 
has  important  implications  in  the 
accurate  prediction  of  the 
aerodynamic  heating  on 
hypersonic  vehicles 


and  thrust  in  propulsive  nozzles.  In 
turn,  nonequilibrium  vibrational 
energy  distributions  are  required  for 
prediction  of  dissociation  rates, 
interpretation  of  radiation 
experiments,  and  interpretation  of 
ionic  recombination  rates. 

At  high  temperatures,  molecular 
collisions  result  in  the  exchange  of  the 
translational,  rotational,  vibrational, 
and  electronic  energies  with  the 
collision  partners.  The  probabilities  of 
these  elementary  processes  differ 
significantly,  giving  rise  to  widely 
separate  relaxation  times  for  the 
internal  modes.  Thus,  it  becomes 
important  to  account  for  the  rates  of 
relaxation  processes  to  predict  the 
nonequilibrium  behavior.  Vibrational 
equilibration,  in  particular,  involves 
widely  separated  relaxation  times, 
ranging  from  the  very  short  times  for 
translational/rotational  equilibration  to 
the  longer  times  for  chemical  and 
ionization  equilibration.  The  disparate 
eigenvalues  of  these 
relaxation  times  have 
significant  physical  and 
computational  implications. 

Continued  Next  Page- 


Figure  1.  Systems  representing 
external  (X-15,  above)  and  internal 
flows  (Rocket  nozzle,  right)  in  thermal 
nonequilibrium  (images  taken  from  http;//www.grc.nasa.gov/ 
WWW/K-1 2/airplane/rocket.html). 
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Methodology 

The  most  fundamental  mathematical 
description  of  modeling  the  kinetic 
processes  associated  with  the  internal 
energy  excitation  of  the  vibrational 
and  rotational  modes  is  given  by  the 
master  rate  equations. 

These  time-dependent  differential 
equations  account  for  the  population 
distributions  in  the  quantum  energy 
states  by  considering  the  kinetics  of 
particle  exchanges  and  can  thus 
account  for  non-Boltzmann  distributions 
that  do  not  assume  a  mean  energy  for 
the  vibrational  modes. 

Figure  2  shows  a  schematic  of  the 
potential  energy  well  depicting  the 
various  kinetic  processes  of  vibration- 
translation,  vibration-vibration,  and 
dissociation  transfers  that  could 
occur  as  a  result  of  vibrational 
energy  excitation. 

The  master  rate  equations  are 
coupled  to  macroscopic  fluid  dynamic 
equations  to  form  a  set  of  conservation 
equations:  mass,  momentum,  and 
energy  in  the  flowfield.  The  mass 
conservation  equations,  however,  are 
written  for  each  quantum  level  and 
thus  give  rise  to  a  very  large  and 
mathematically  complex  set  of  partial 
differential  equations. 

The  number  of  quantum  levels 
depends  on  the  energy  of  dissociation 
of  the  molecule.  For  example,  the  two 
principal  constituents  of  air — nitrogen 
and  oxygen — require  47  and  33 
quantum  levels  respectively. 

For  a  nitrogen-nitrogen  collision, 
master  rate  equation  calculations 
account  for  the  energy  exchange 
processes  amongst  the  47  quantum 
energy  levels  of  the  nitrogen  molecule. 

These  calculations  determine  the 
probability  of  a  specific  process 
contributing  to  the  dissociation 
process.  As  an  example,  the 
probability  of  vibration-vibration 
energy  transfers  in  a  nitrogen  gas 


maintained  at  5000°  Kelvin  (K) 
contributing  to  the  dissociation 
process  is  depicted  in  Figure  3. 

The  axes  in  Figure  3,  represented  by  i 
and  j,  denote  the  quantum  levels  in 
nitrogen  molecule  A  colliding  with 
nitrogen  molecule  B.  As  seen  in  the 
figure,  the  high  energy  exchange 
probability  is  primarily  in  the  lower 
levels  of  the  vibrational  manifolds  of 
both  colliding  molecules.  The 
mathematical  complexity  of  the  non¬ 
linear  governing  partial  differential 
equations  requires  concomitantly 
advanced  numerical  methods. 
Consequently,  the  Roe  flux-difference 
split  method  is  employed  to  solve  the 
coupled  set  of  equations.  The  method 
is  implemented  in  a  finite  volume 
formulation  by  computing  the  cell 
interface  fluxes. 

The  mass  conservation  equation  is 
written  for  mass  density  in  each 
quantum  level.  The  source  term  of 
production  and  destruction  of 
“species”  derived  from  the  vibrational 
master  equations  is  composed  of  the 
relevant  energy  exchange  processes, 
consisting  of  the  V-T  and  V-V 
reaction  mechanisms. 

Solutions 

Vibrational  Nonequilibrium 

The  distribution  of  computed 
vibrational  levels  for  a  Mach  6.5 


nitrogen  flow  past  a  forebody  is  shown 
in  Figure  4. 

The  plots  show  fractional  density 
versus  vibrational  quantum  level  energies 
with  the  corresponding  quantum 
numbers  identified  along  the  curves. 
Population  distributions  versus  quantum 
level  vibrational  energies  are  plotted 
along  four  select  locations  along  the 
stagnation  streamline. 

The  four  locations  are  Stream,  Pre-Shock, 
Post  Shock,  and  Stagnation  Point  (or 
Surface).  The  Stream  location  has 
freestream  conditions  and  is  several 
grid  points  before  the  onset  of  the 
bow  shock;  Pre-Shock  (at  the  shock 
edge)  is  at  the  grid  point  immediately 
before  the  highest  temperature  and 
pressure  values  across  the  shock;  and 
the  Post- Shock  location  is  the  point 
immediately  behind  the  shock-front  in 
the  shock  layer.  Figure  4  also  shows 
that  the  population  distribution  is 
Boltzmann  in  the  freestream  and  non- 
Boltzmann  in  the  shock  layer. 

Vibrational  Excitation  Leading 
TO  Dissociation 

The  study  of  the  kinetics  of  thermal 
dissociation  demonstrates  that  the 
dissociation  process  alters  the 
vibrational  distribution  function.  As 
equilibrium  is  approached,  a  quasi¬ 
steady  state  is  achieved,  and  the 
populations  obtain  a  self-similar  form. 


Figure  2.  Energy  transfers  in  the  molecular  vibrational 
manifold  depicting  energy  exchange  processes  that 
occur  over  a  wide  range  of  time  scales. 
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Figure  3.  Probability  of  vibration- 
vibration  energy  transfers  among 
the  quantum  states  in  nitrogen- 
nitrogen  collision  at  5000^  K. 

The  dissociation  losses  from  the 
vibrational  manifold  tend  to  reduce 
the  vibrational  populations  below  their 
equilibrium  values,  resulting  in  the 
lowering  of  the  dissociation  rate.  At 
equilibrium,  the  net  loss  due  to 
dissociation  and  recombination  is 
zero,  and  there  is  no  depletion  of 
the  population. 

A  state-to-state  kinetic  description  of 
the  nonequilibrium  process  allows  the 
researcher  to  develop  the  internal 
energy  mode  coupling  models,  such 
as  the  vibration-dissociation  coupling 
model,  essential  to  the  understanding 
of  the  aerothermodynamics  of 
aerospace  vehicles. 

It  is  convenient  to  divide  the  vibrational 
relaxation  into  two  time  regimes. 
Initially,  over  a  time  t  ^  quasi-steady 
distribution  is  established.  Over  a  much 
longer  time  t  this  distribution 
approaches  the  equilibrium  distribution, 
and  the  atoms  and  molecular  species 
concentrations  then  ultimately  satisfy 
the  Law  of  Mass  Action.  Here,  the 
evolution  of  the  quasi-steady 
distribution  as  it  evolves  in  time  toward 
the  equilibrium  distribution  is  considered. 


In  the  case  of  t  VT  -  <  f  <  t  two 
opposing  factors  characterize  this 
state:  V-T  transfers  seek  to  establish 
equilibrium,  while  dissociation  disrupts 
and  perturbs  the  distribution. 

The  nonequilibrium  conditions  behind 
the  shock  wave  of  the  blunt  body 
example  discussed  in  Figure  4  exemplity 
the  case  where  the  translational 
temperature  is  greater  than  the 
vibrational  temperature,  ultimately 
approaching  near-equilibrium  conditions 
close  to  the  surface  of  the  body. 

The  higher  rate  of  dissociation  versus 
recombination  in  blunt  body  flows 


makes  the  vibrational  population 
depletion  effect  significant  in  the 
vibration-dissociation  coupling  model. 
However,  in  expanding  nozzle  flows, 
which  are  aitical  for  thmst  development, 
the  effect  of  vibration-dissociation 
coupling  on  the  vibrational  population 
density  is  reversed,  and  the  population 
is  enhanced. 

For  sonic  flows  at  the  inlet  of  an 
expansion  nozzle,  the  flow  is  in  thermal 
equilibrium,  and  as  it  proceeds  towards 
the  exit  of  the  nozzle,  the  low  departs 
from  the  equilibrium  conditions.  The 
vibrational  temperature  freezes  near 
the  inlet,  the  translation  temperature 
rapidly  falls  as  a  result  of  the  expansion 
process,  and  the  flow  exhibits  higher 
degrees  of  nonequilibrium  as  it  proceeds 
towards  the  exit.  As  the  translational 
temperature  starts  dropping,  the 
dissociation  rate  drops  significantly. 

However  for  the  same  translational 
temperature  drop,  the  change  in  the 
recombination  rate  is  small,  and  the 
recombination  rate  remains  consistently 
higher  than  the  dissociation  rate 
throughout  most  of  the  nozzle  flowfield. 

Continued  Next  Page- 


Figure  4.  Nonequilibrium  population  distribution  at 
select  locations  in  a  flow  past  a  vehicle  forebody, 
with  Mach  6.5  nitrogen  flow  and  a  1 -meter 
forebody  radius.  The  abscissa  denotes  the 
vibrational  energy  distribution  in  the  quantum 
energy  states  of  nitrogen. 


NAVO  MSRC  NAVIGATOR 


SPRING  2004 


11 


This  recombination-dominant  flow 
leads  to  a  population  enhancement  in 
the  vibrational  manifold  under  the 
nonequilibrium  reactive  conditions.  A 
vibration-dissociation  coupling 
model  thus  has  to  account  for  this 
enhancement  to  accurately  predict 
the  nonequilibrium  chemistry.  Figure 
5  shows  the  population  depletion 
and  enhancement  in  high-speed 
flow  situations. 


The  proper  simulation  and  understanding 
of  the  thermodynamic  nonequilibrium 
processes  occurring  in  the  flowfields  of 
the  high  speed  systems  are  crucial  for 
prediction  of  aeroheating  and  drag  in 
external  flows  around  vehicles  and 
thrust  in  the  internal  flows  of 
propulsive  nozzles. 

These  simulations  require  solving  a 
mathematically  complex  set  of  non¬ 
linear  governing  partial  differential 


equations.  The  solution  of  these  sets 
is  computationally  intensive  and 
requires  the  computational  capacity 
provided  by  High  Performance 
Computing  architectures. 


Population  Depletion  in  Upper  Vibrational  Levels  in  O2  Dissociation 


High  Population  Enhancement  in  O2  Flow  Neat  Nozzle  Exit 
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Figure  5.  Vibrational  population  depletion  and  enhancement  signifying  departure  from  equilibrium  conditions  in 
high-speed  flows,  (a)  flow  past  a  vehicle  forebody,  (b)  flow  inside  an  expansion  nozzle  for  equilibrium  throat 
temperature  of  5000^  K  and  exit  translational  temperature  of  2548^  K  and  exit  vibrational  temperature  of  2761^  K. 
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NAVO  MSRC  Visualization:  Open 
Source  Solutions 

Sean  Ziegeler,  NAVO  MSRC  Visual  Analysis  and  Data  Interpretation  Center  (VADIC) 


Open  source  and  free  software  products 
are  gaining  significant  popularity  in 
business  and  government.  Open 
source  products  have  several 
advantages  that  compel  organizations 
to  consider  them  as  viable  alternatives 
to  commercial  software. 

In  most  cases,  the  greatest  advantages 
are  that  open  source  software  is  free 
— which  can  result  in  significant 
savings  and  is  hardware  platform 
independent — which  enables  users  of 
open  source  software  to  select  the 
most  cost-effective  hardware  for  their 
application. 

In  addition,  applications  developed 
using  open  source  software  can  be 
redistributed  without  cost.  So,  in 
addition  to  having  no  cost  to  use,  this 
results  in  another  significant  and 
attractive  advantage:  there  will  not  be 
any  licensing  fees  to  distribute  an 
application  based  on  open  source  code. 
At  the  NAVO  MSRC  Visual  Analysis 
and  Data  Interpretation  Center 
(VADIC),  these  aspects  of  open  source 
tools  have  enabled  the  production  of 
higher  quality  and  more  affordable 
products  for  VADIC  clients.  This 
article  uses  the  example  of  the  VADIC 
work  for  the  National  Aeronautics  and 


Space  Administration  (NASA)  Goddard 
Space  Flight  Center  on  Sea  Surface 
Temperature  (SST)  images  to  demonstrate 
the  open  source  advantages. 

The  Open  Source  Advantage 

At  the  heart  of  the  open  source 
software  phenomena  is  that  small 
groups  of  enthusiasts  can  evolve  into 
a  worldwide  collaborative  effort. 

This,  in  turn,  fosters  tremendous 
growth  in  the  software,  which  leads  to 
innovation  and,  in  the  end, 
culminates  in  full-featured  software 
that  can  compete  with  its  commercial 
counterparts.  One  of  the  most 
convincing  examples  of  this  is  Linux, 
which  began  as  one  person's  hobby. 


and  has  become  a  practical,  powerful 
operating  system  with  hundreds  of 
contributors  and  even  more  users 
around  the  world.  Similarly,  many 
organizations  utilize  the  popular 
Apache  software  to  manage  their  Web 
servers,  and  many  information 
systems  are  implemented  using 
MySQL,  a  popular  database 
management  tool.  In  fact,  the  tools 
range  in  complexity  from  entire 
compilers,  such  as  GNU  C+  +  ,  to  Web 
browsers  like  Mozilla. 

Transitioning  to  open  source  products 
usually  involves  an  evaluation  period, 
when  the  legacy  application  and  the 
new,  open  source-based  application 

Continued  Page  25... 


Figure  1 .  (Above)  Sea  Surface 
Temperature  (SST),  ranging  from 
cold  (blue)  to  warm  (red). 


Figure  2.  (Left)  SST  Anomaly, 
ranging  from  below  average  (blue) 
to  above  average  (red). 
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Jets  of  fluid  issuing  into  a  cross- 
flow  of  fluid  Jets  in  cross-flow 
occur  in  a  number  of  different 
applications  such  as  film  cooling 
for  turbine  biades,  thrust  and 
noise  control  of  Vertical  Take  Off 
and  Landing  (VTOL)  aircraft, 
fuel-air  mixing  in  gas  turbine 
combustors,  and  poiiutant 
dispersion  from  chimney  stacks. 

Experiments  with  non-reacting 
flows  have  shown  that  pulsing 
the  Jet  can  have  a  iarge  effect  on 
the  mixing,  spreading,  and 
penetration  of  the  free  jet. 
Therefore,  external  modulation  of 
the  Jet  is  a  potentiai  strategy  for 
enhancement  of  the  mixing 
effectiveness  of  jets  in  cross-flow. 

This  improvement  in  fuel-air 
mixing  ieads  to 
better 


overaii 
performance 
in  a  variety  of 
applications 
(e.g.,  combustion). 

Direct  Numerical  Simulations 
AND  Tetra 

The  goal  of  the  present  study  is  to 
computationally  analyze  the  effect 
of  external  mocfulation  (pulsing) 
on  the  flow  structures  and  the 
resulting  mixing  of  a  Jet  in  cross- 
fiow. 

As  the  flow  is  inherently 
unsteady  (due  to  the  external 
pulsing),  the  key  to  the  successful 
prediction  of  such  flows  is  the 
ability  to  resolve  the  dynamics  of 
all  important  flow  structures  that 
result  from  the  interaction  of  the 
unsteady  pulsed  Jet  with  the  main 
flow  in  the  combustor. 

In  addition,  to  understand  why 
external  excitation  changes  the 
flow,  it  is  necessary  to 
resolve  and  track  the 
different  dynamic 


flow  structures 
resulting  from  different 
external  excitations. 

Asa  result,  time-averaged 
Reynolds  Averaged  Navier-Stokes 
solutions  provide  very  little 
insight  into  the  issues  of  interest 
in  the  present  work.  Therefore,  a 
Direct  N umeri cal  Simulation 
(DNS)  is  performed,  and  the 
results  are  used  to  examine  the 
effects  of  forcing  on  the  flow 
structures  and  mixing.  To  aid  in 
quantifying  the  mixing,  an 
equation  governing  the  evolution 
of  a  passive  scalar  is  solved  along 
with  the  incompressible  Navier- 
Stokes  equations. 

Another  reason  for  performing  a 
D  N  S  of  this  flow  (as  opposed 
to  an  experimental  study) 
is  that  a  computational 
study  can  provide 
information  that  is 
very  difficult  to 


obtain  in  an  experimental  study. 

In  the  present  work,  massless 
particles  are  released  into  the  flow 
at  various  locations. 

These  particles  are  colored  by 
their  seed  locations  and  residence 
time,  greatly  aiding  the 
understanding  of  the  dynamics  of 
the  flow.  Unlike  in  experiments, 
these  particles  can  be  introduced 
at  arbitrary  locations  without  the 
detrimental  effect  of  the  presence 
of  physical  seeding  apparatus 
influencing  the  flow. 

In  this  work,  a  single  jet  in  a 
cross-flow  is  studied.  The 
Reynolds  number  based  on  UJet 
and  Jet  diameter  (d)  is  5000, 

Continued  Next  Page 


while  the  blowing  ratio,  defined  as 
Ujet  divided  by  the  maximum  cross- 
flow  inlet  velocity,  is  6.  The  jet  is 
injected  at  90  degrees  to  the  cross- 
flow.  The  scalar  at  the  jet  exit  is  set 
to  one,  while  at  the  cross-flow  inlet, 
it  is  set  to  zero.  Referring  to  Figure  1, 
Ly  =  17d,  Lp  =  10d,  L^p=7d,  and 
Ldown^25d.  Solutions  are  obtained 
on  three  different  grids  that  contain 
2,  15,  and  120  million  grid  points. 
The  time  step  used  results  in 
maximum  Courant-Friedrich-Levy 
(CFL)  numbers  of  approximately  .3 
at  any  point  in  the  domain. 

After  reaching  a  statistically  steady 
state,  turbulent  statistics  are  collected 
at  each  time  step  until  the  flow 
(based  on  the  cross-flow  velocity) 
has  traveled  a  distance  equal  to  534 
jet  diameters  (d).  As  statistics  are 
collected  at  every  time  step,  this 
results  in  statistics  being  averaged 
over  160,201  distinct  realizations  of 
the  flow  field. 


Solutions  to  this  flow  problem  are 
obtained  by  the  use  of  the  Tetra^ 
computer  program  developed  by 
Frank  Muldoon  at  Louisiana  State 
University  (LSU). 

Tetra  solves  the  incompressible 
Navier-Stokes  equations  as  well  as 
equations  governing  the  transport  of 
passive  scalars.  It  can  also  efficiently 
track  numerous  massless  particles 
(up  to  20  million  have  been  used  in 
these  simulations). 

In  Tetra,  a  spatially  sixth-order 
accurate  central  difference  scheme 
with  a  monotonic  limiter  is  used  for 
the  convective  terms,  while  a  fourth- 
order  accurate  central  difference 
scheme  is  used  for  the  diffusive 
terms.  The  convective  and  diffusive 
terms  are  integrated  in  time,  using 
an  explicit  third-order  accurate 
method,  while  the  colored 
Symmetrically  Coupled  Gauss-Seidel 
method  is  used  to  solve  the  Navier- 
Stokes  equations. 


Tetra  is  written  in  FORTRAN  95  and 
parallelized  using  the  Message 
Passing  Interface  (MPI).  Tetra  has 
been  run  on  parallel  machines  such 
as  the  CRAY  T3E,  SGI  Origin  3000, 
Helix  (Linux  cluster  at  LSU),  and  an 
IBM  SP4  (MARCELLUS)  at  the 
NAVO  MSRC. 

The  simulations  in  this  article  were 
run  on  MARCELLUS  using  8  to  16 
processors  for  the  15  million  and 
128  processors  for  the  120  million 
grid  point  grids.  Based  on  wall  clock 
time,  the  120-million  grid  point 
simulation  attains  2.5  x  lO-^  seconds 
per  time  step  per  grid  point. 

Strouhal  Numbers  and  Waveforms 

The  forcing  frequency  can  be 
characterized  by  a  dimensionless  jet 
Strouhal  number  (kd/Ujet),  where  k 
is  the  forcing  frequency  in  Hertz 
(Hz).  In  the  present  work,  the 
modulation  amplitude  is  such  that 
the  instantaneous  mass  flow  through 
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Figure  1.  Problem  schematic. 
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the  jet  exit  is  always  between 
80  and  120  percent  of  the  time- 
averaged  mean  mass  flow  through 
the  jet  exit. 

In  the  course  of  this  work,  a  range 
of  Strouhal  numbers  (.025,  .05,  .1, 
.2,  .4,  .6,  .8,  and  1)  were  examined 
for  sine,  square,  and  spiked 
waveforms  on  the  coarsest  grid.  This 
was  done  to  inexpensively  identify 
combinations  of  Strouhal  numbers 
and  waveforms  for  which  there  are 
significant  changes  in  the  flow. 

As  dramatic  changes  in  the  mixing, 
spreading,  and  penetration  of  the  jet 
were  observed  at  Strouhal  numbers 
of  .2,  .4,  and  .6  for  the  sine  wave 
forms,  a  DNS  is  performed  for  these 
cases  along  with  the  unpulsed  case. 

In  this  study,  pulsing  at  a  Strouhal 
number  of  .4  causes  the  jet  to  split 
into  three  vertical  jets  and  to 
penetrate  much  further,  while  a 


Strouhal  number  of  .6  causes  the  jet 
to  split  into  two  horizontal  jets  and 
to  remain  much  closer  to  the  jet  exit 
(see  Figure  2).  Pulsing  at  a  Strouhal 
number  of  .2,  however,  causes  the 
jet  to  split  into  two  vertical  jets. 

In  addition,  it  has  been  found  that 
the  merging  of  the  shear  layer 
vortices  at  the  leading  edge  of  the 
jet  occurs  at  Strouhal  numbers  of  .2 
(see  Figure  3)  and  .6. 

At  a  Strouhal  number  of  .6,  this 
merging  occurs  at  approximately  the 
same  location  that  the  jet  bifurcates 
in  the  horizontal  direction  and  is 
likely  related  to  the  bifurcation.  For 
pulsed  cases  the  frequency  of  the 
shear  layer  vortices  which  form  at 
the  front  of  the  jet  is  that  of  the 
imposed  pulsing,  while  for  the 
unpulsed  case  the  frequency  of  the 
shear  layer  vortices  corresponds  to  a 
Strouhal  number  of  .35. 


Wake  Vortices 

While  it  is  commonly  thought  that 
the  wake  vortices  originate  in  the 
incoming  wall  boundary  layer,  the 
results  of  this  work  show  a  different 
origin  for  the  wake  vortices.  Based 
on  particle  traces,  it  can  be 
ascertained  that  the  wake  vortices 
do  not  contain  any  jet  fluid. 

The  core  of  the  wake  vortices  has  its 
origin  in  fluid  (i.e.,  green  particles) 
at  the  back  of  the  jet  close  to  the 
wall  (see  Figure  4).  This  fluid  is 
entrained  into  the  shear  layer 
vortices  at  the  back  of  the  jet.  Part 
of  this  fluid,  at  the  very  back  of  the 
jet,  is  stripped  off  by  the  cross-flow 
and  forms  the  seed  or  core  of  the 
wake  vortex. 

As  the  wake  vortex  moves 
downstream,  it  grows  in  diameter  as 
cross-flow  fluid  (yellow  particles). 

Article  Continues  Next  Page 


Figure  2.  0.1  Iso-surfaces  of  scalar  colored  by  magnitude  of  gradient  of  scalar  (a)  Strouhal  =.6,  (b)  Strouhal  =  .4. 
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Figure  3.  Shear  layer  vortex  merging  at  Strouhal  =.6,  particles  colored  by  residence  time. 


originating  upstream  of  the  jet,  wraps 
around  the  seed  or  core  of  the  vortex. 
As  the  wake  vortices  move 
downstream,  they  feed  fluid  from 
the  cross-flow  into  the  jet.  The  center 
of  the  vortices  continues  to  be 
dominated  by  fluid  originating  at 
the  back  of  the  jet  close  to  the  wall. 
This  fluid  travels  along  the  center  of 
the  wake  vortices  in  a  corkscrew 
fashion  upward  into  the  jet.  It  is 
surrounded  by  fluid  originating 
upstream  of  the  jet,  which  also 
travels  in  a  corkscrew  fashion 
upward  into  the  jet,  representing  a 
mechanism  by  which  the  jet  is  diluted 
by  the  surrounding  flow.  Pulsing  at  a 
Strouhal  number  of  .6  completely 
suppresses  the  wake  vortices. 

Symmetry  and  Asymmetry  in 
Flow  Models 

Though  it  is  often  assumed  that 
symmetry  will  exist  in  the  time- 
averaged  results,  this  may  not  be 


the  case.  Asymmetry  in  the  cross¬ 
stream  plane  has  been  found  in  a 
few  experimental  works  but  has  not 
been  extensively  studied. 

While  it  is  impossible  in  an  experiment 
to  remove  all  asymmetries  regarding 
elements  such  as  the  incoming  flow, 
the  physical  geometry  of  the  jet 
nozzle,  etc.,  much  greater  control 
exists  in  computations. 


In  the  present  work,  there  are  two 
sources  of  asymmetry:  round-off 
error  and  symmetry  breaking  in  the 
initial  conditions  (used  to  hasten 
transition  to  turbulence). 

The  round-off  error  is  a  source  of 
asymmetry  because,  while  the  same 
numbers  are  in  a  finite  difference 
stencil  on  both  sides  of  the  z-axis, 
the  numbers  are  added  together  in  a 


Figure  4.  Wake  vortex  formation,  at 
Strouhal  number  of  .4. 
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different  order  on  each  side  of  the  z- 
axis.  No  asymmetry  is  introduced  by 
the  boundary  conditions,  as  they  are 
symmetrical  about  the  z-axis. 


The  asymmetry  caused  by  the 
round-off  error  is  very  small,  while 
the  asymmetry  resulting  from  the 
initial  conditions  should  decay  (in  a 
time-averaged  sense)  if  physically  an 
asymmetry  does  not  exist.  In  the 
present  work,  asymmetry  exists  only 


Bzr^?  3  -I  5 


Figure  5.  Asymmetric  states, 
unpulsed  jet  (a)  symmetry 
breaking  in  initial  conditions, 
(b)  symmetric  initial  conditions. 


for  the  unpulsed  jet  and  for  sine  wave 
pulsing  at  a  Strouhal  number  of  .2. 
Figure  5a  shows  the  results  of 
symmetry  breaking  in  the  initial 
conditions,  while  Figure  5b  shows 
the  result  of  symmetric  initial 
conditions.  It  can  be  seen  that  the 
flow  can  assume  either  of  these  two 
apparently  mirror  image  states. 

Therefore,  it  can  be  concluded  that  for 
cases  in  which  asymmetry  is  seen,  the 
flow  is  very  susceptible  to  asymmetries 
and  that  one  of  the  two  asymmetric 
states  would  appear  in  an  experiment 
in  which  asymmetries  in  the  initial  and 
boundary  conditions  would  likely  be 
larger  than  the  computations. 


Conclusion 

In  conclusion,  by  analyzing  the 
results  of  a  DNS,  a  new  origin  has 
been  found  for  the  formation  of 
wake  vortices,  while  at  certain 
pulsing  frequencies  the  wake  vortices 
can  be  suppressed.  Also,  a  significant 
asymmetry  that  primarily  affects  the 
wake  vortices  has  been  found  for 
certain  cases. 

The  finding  of  this  asymmetry,  which 
can  occur  in  one  of  two  mirror  images, 
confirms  that  found  in  the  experimental 
works  of  Narayanan,  Barooah  et  al.2 
and  Smith  and  Mungal.^ 

In  the  future,  work  on  this  question 
may  include  the  storage  of  complete 
time  sequences  of  data,  enabling 
interactive  seeding  and  tracking 
of  particles. 

One  question  that  this  investigation 
will  help  answer  is  whether  the 
branches  of  the  split  jet  (resulting 
from  pulsing)  preferentially  contain 
fluid  exiting  the  jet  from  certain 
regions  of  the  pulsing  wave.  Also 
under  consideration  is  the  coupling 
of  an  optimization  algorithm  to  Tetra 
to  enable  a  more  rational  approach 
to  determine  types  of  pulsing  resulting 
in  improved  mixing  and/or  penetration. 
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NAVO  MSRC  PET  Update 

Eleanor  Schroeder,  NAVO  MSRC  Programming  Environment  and  Training  Program 
(PET)  Government  Lead 

This  marks  the  beginning  of  the  first  option  year  of  the 
PET  contract.  Hopefully  most  of  you  who  have  interests 
that  fall  within  PET's  scope  have  accounts  on  the  Online 
Knowledge  Center  (OKC).  The  pages  on  that  Web  site 
underwent  changes  during  the  third  contract  year,  and 
hopefully  you  have  discovered  that  information  is  easier  to 
find  now.  Of  course,  if  you  still  have  suggestions  for 
improvement,  we  are  always  happy  to  hear  them. 


Year  three  did  introduce  some  other  changes  as  well.  We 
brought  on  two  additional  people  to  augment  core 
support  for  the  Computational  Environments  (CE) 
functional  area  (FA).  We  did  this  to  ensure  a  smooth 
transition  of  the  Consistent  Well-documented 
Computational  Environment  project  into  the  core  support 
arena.  So  I'd  like  to  welcome  both  Brett  Ellis  and  Joseph 
Thomas,  both  located  at  the  University  of  Tennessee  in 
Knoxville,  to  PET. 

In  the  world  of  Climate,  Weather,  and  Oceanography 
(CWO),  we  had  difficulty  finding  a  good  candidate  to  fill 
the  vacant  onsite  position  at  Engineer  Research  and 
Development  Center  (ERDC).  As  a  result,  we  opted  to  do 
a  temporary  fix  and  are  allowing  the  University  of  Texas, 
Austin,  to  hire  a  person  to  be  a  “roving  onsite”  for  the 
next  six  months  until  we  can  find  a  permanent 
replacement  to  be  located  at  a  DoD  installation.  Of  course, 
we  still  have  our  two  extremely  capable  onsites.  Dr.  Tim 
Campbell  at  Stennis  Space  Center,  MS,  and  Dr.  John 
Romo  in  Monterey,  CA. 

Projects  are  still  an  integral  of  the  PET  program  for  this  first 
option  year  of  the  contract.  Project  selections  have  been 
made,  and  detailed  information  can  be  found  on  the  OKC. 

In  PET  Component  1,  we  are  continuing  the  support  we 
have  been  providing  to  the  Weather  Research  and 
Forecast  (WRF)  model,  expanding  the  effort  to  integrate 
WRF  into  the  Advanced  Climate  Modeling  and 
Environmental  Simulations  Program  (ACMES)  based  at 
the  Air  Force  Combat  Climatology  Center  (AFCCC) .  This 
should  greatly  benefit  the  AFCCC’s  operational 
effectiveness  as  well  as  benefit  the  many  customers  who 
utilize  ACMES  data  in  DoD  operational  activities. 

The  University  of  Hawaii,  a  Minority  Serving  Institution 
(MSI),  was  also  awarded  a  project  involving  tropical 
cyclone  forecasting.  Planned  enhancements  to  the  Naval 
Operational  Global  Atmospheric  Prediction  System 
(NOGAPS)  should  improve  the  current  NOGAPS  tropical 
cyclone  forecast  skill. 


We  are  looking  at  providing  improved  linear  and 
nonlinear  solvers  for  environmental  quality  modeling 
(EQM)  codes.  The  expected  outcome  of  this  effort  will  be 
a  set  of  semi-generic  scalable,  parallel  solver  modules  that 
have  been  well-documented  to  allow  the  DoD  users  in 
EQM  and  similar  areas  to  easily  implement  these  solvers 
into  their  codes,  thus  improving  the  efficiency  of  the 
codes'  compute  times. 

CE,  PET  Component  I's  cross-cutting  FA,  is  looking  at 
four  different  issues.  Two  will  be  providing  support  to 
benchmarking/profiling  of  application  data.  One  project  is 
looking  at  automated  collection  of  performance  data  into 
a  database  framework  that  will  also  offer  a  Java-based 
Application  Program (ming)  Interface  (API)  for  performance 
analysis  tools.  The  second  effort  will  deploy  and  document 
a  set  of  end-user  tools  for  statistical  analysis  of  large-scale 
performance  data. 

The  Scalability  and  Performance  Optimization  Team 
(SPOT),  an  effort  started  in  the  third  contract  year  that 
is  similar  to  the  Tiger  Teams  from  the  first  PET  program, 
will  continue  their  work  into  this  first  option  year.  DoD 
users  who  have  codes  that  they  believe  would  benefit  from 
this  type  of  effort  are  encouraged  to  contact  the  SPOT 
members,  myself,  and/or  their  Computational  Technology 
Area  (CTA)  lead  to  get  assistance. 

CE  will  also  be  delving  into  the  area  of  dynamic  process 
and  fault-tolerance  support  for  MPI-based  varying  load 
DoD  applications.  Concentration  will  be  primarily  on 
Signal  and  Image  Processing  applications  this  year,  but 
we  hope  that  the  results  will  be  extensible  to  other 
interested  applications. 

We  have  our  work  cut  out  for  us  this  year,  but  PET 
Component  1  has  an  extremely  well-qualified  and 
enthusiastic  group  of  partners  that  will  perform  this  work. 
As  their  technical  advisor,  I  am  always  open  to  hearing 
suggestions  from  you,  our  user  community  and  customer 
base,  for  what  we  can  do  to  further  improve  PET  to  better 
meet  your  needs. 
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Navigator  Tools  and  Tips 


User  Data  Storage  Management 


Sheila  Carbonette,  NAVO  MSRC  User  Support 

In  the  last  few  months,  the  total  data  archived  on  the  Mass 
Storage  Servers,  JULES  and  VINCENT,  have  increased  to 
over  900  Terabytes  (TB). 

The  total  number  of  archived  user  directories  and  files  is  in 
the  millions  and  growing  rapidly.  Maintaining  superfluous 
files  or  creating  an  overly  complex  directory  structure  will 
unnecessarily  degrade  performance.  Therefore,  we,  the 
user  community,  should  remove  unnecessary  files  and 
streamline  our  directory  structure.  By  practicing  prudent 
storage  practices,  we  can  improve  the  overall  performance 
of  the  storage  management  system. 

During  the  last  quarter,  MSRC  User  Support  sent  a  notice 
announcing  the  availability  of  monthly  data  summary 
reports  to  help  users  manage  their  archived  data.  Each 
monthly  report  will  be  stored  as  an  ASCII  file  named 
mas_store_  usage  in  each  user  home  directory  on  the 
Archive  Server.  The  mas_store_usage  file  for  each  user  will 
be  updated  on  the  first  day  of  each  month.  For  example: 

{jules:/u/honne/shecar}[l]%  Is  -I 
nnas_store_usage- 

rw-r--r--  1  shecar  other  166637  M  ar 

1  00:16  nnas_store_usage 

Each  directory  and  subdirectory  under  your  login  account 
are  listed  in  this  report,  with  the  number  of  files  and  data 
storage  used  per  directory  broken  out  by  access  age.  The 
example  below,  lists  the  first  line  of  the  summary  report  file 
for  user  shecar: 

jules:/u/home/shecar}[2]%  head  -1 
;mas_store_usage 

Ihecar  64f  154.007GB  (A/M  :  <ly)  57f 
120. 007GB  (A/M  :  ly-<2y) 

O.^OGB  (A/M^  2y-<5y)  lf^34J300GB _ 

The  directory  shecar  contains  a  total  of  64  files  that 
consume  154  Gigabyte  (GB)  of  space.  Of  these  64  files, 

57  files  are  less  than  one  year  old  and  take  up  120  GB  of 
space.  There  are  six  files  that  are  between  one  and  two 
years  old  taking  up  less  than  1  GB  of  space.  Lastly  there  is 
one  file,  taking  up  34  GB  of  space,  that  has  not  been 
accessed  in  more  than  two  years. 

You  can  also  determine  the  total  amount  of  storage  that 
you  have  used  with  the  SAM/FS  command  sdu.  This 
command,  located  in  the  /opt/SUNWsamfs/bin  directory,  is 
the  SAM  equivalent  of  the  UNIX  du  command. 


However,  the  sdu  command  includes  both  disk  and  tape 
storage  media.  For  example: 

jules:/u/home/shecar}[3]%  sdu  -sk  . 

1782983 

ules:/u/honne/shecar}[4]%  du  -sk  . 

157508  . 

In  addition.  User  Support  will  contact  users  whose  data 
holdings  on  the  Archive  Servers  exceed  1  TB  and/or 
100,000  files.  This  communication  will  request  users  to 
determine  if  nonessential  files  and  directories  can  be 
deleted  or  consolidated.  Again,  a  reduction  in  storage  use 
or  a  consolidation  of  the  directory  structure  will  result  in  a 
performance  improvement  by  saving  archival  server 
storage  space  and  filesystem  inodes. 

The  NAVO  User  Support  group  recognizes  that  data  are 
critical  to  the  users  of  the  NAVO  MSRC.  Without  data, 
there  are  no  solutions  to  the  problems  being  researched  on 
the  MSRC  resources.  In  fact,  without  data  there  is  no  need 
for  HPC  resources. 

We  also  understand  that  as  the  performance  and 
capabilities  of  the  HPC  resources  grow,  the  complexity  of 
the  problems  solved  on  these  resources  also  grows,  which 
drives  a  corresponding  growth  in  the  data  required  and 
generated  by  solving  these  complex  problems.  Our  interest 
is  to  ensure  that  we  are  using  our  data  storage  capacity 
efficiently  and  wisely.  With  an  emphasis  on  optimal  file 
management  of  their  archived  data,  all  NAVO  users  will 
benefit  from  a  streamlined  server  storage  structure  with 
decreased  overall  system  resource  requirements. 

If  there  are  any  questions  or  suggested  improvements  for 
the  summary  reports,  please  contact  the  NAVO  MSRC 
User  Support  at  1-800-993-7677  or  via  E-mail  to 
msrchelp@navo.hpc.mil. 
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Changing  Kerberos  Passwords 


NAVO  MSRC  User  Support 

During  the  last  quarter,  NAVO  MSRC  implemented 
password  aging  on  Kerberos  passwords  for  all  users  homed 
at  the  NAVO  MSRC.  Users  are  now  required  to  change 
their  passwords  every  90  days.  The  passwords  created 
should  be  unique  and  secure  and  must  be  at  least  eight 
characters  and  contain  three  of  the  following  characteristics: 
upper-case  letters,  lower-case  letters,  and  numbers/ 
special  characters. 

Users  who  create  Kerberos  tickets  on  a  daily  basis  with 
kinit  or  through  the  krb5.exe  application  will  receive  a 
message  that  their  password  is  about  to  expire.  These 
expiration  messages  will  begin  within  three  weeks  of  the 
actual  password  expiration  date  and  will  continue  until  the 
password  actually  expires.  This  message  will  be  displayed 
every  time  a  user  creates  an  initial  Kerberos  ticket. 

Users  who  use  only  the  Information  Environment  (IE)  will 
not  receive  this  expiration  message.  Until  a  solution  is 
implemented.  User  Support  will  notify  these  users  of  their 
password  expiration  date  and  offer  any  assistance  in 
changing  their  passwords.  Below  are  instructions  for 
changing  passwords  on  a  UNIX  platform  as  well  as  a  PC. 

Changing  Passwords  on  UNIX  or  LINUX  Platforms 

To  change  your  Kerberos  password  on  a  workstation 
running  the  UNIX  or  LINUX  Operating  System,  you  must 
execute  the  kpasswd  command  as  shown  below. 

{saturn:/u/honne/shecar}  [1]%  Kpasswd  shecar 

Password  for  shecar@NAVO.HPC.M  IL: 

<ENTER  OLD  PASSW0RD> 

SAM  Authentication 

Chaiienge  for  Security  Dynamics  M  echanism 

SecureiD  Passcode:  <ENTER  GENERATED 
PASSC0DE> 

Enter  new  password:  <ENTER  NEW 
PASSWORD. 

Enter  it  again:  <ENTER  N  EW  PASSWORD 
AGAi  N  > 

Password  changed. 


Changing  Passwords  on  the  Windows 
Operating  System 

To  change  your  Kerberos  password  on  a  workstation 
running  the  Windows  Operating  System,  you  must  first 
execute  the  krb5.exe  application.  Once  the  krb5.exe 
application  is  running,  select  the  Change  Password  button 
at  the  bottom  of  the  dialog  box. 

A  dialog  box  will  appear  prompting  you  to  enter  your 
Name,  Realm,  Old  Password,  and  New  Password  twice. 
After  entering  all  information,  select  the  0  K  button. 

Once  you  have  successfully  entered  your  new  password, 
another  dialog  box  will  appear.  This  box  will  prompt  you 
for  a  SecurlD  passcode. 

If  you  have  already  set  a  Personal  Identification  Number 
(PIN),  generate  a  passcode  using  your  SecurlD  card  and 
your  established  PIN.  Once  the  SecurlD  card  has 
generated  a  passcode,  enter  the  passcode  at  the  Securl  D 
P  asscode  prompt.  Upon  entering  the  code  correctly,  your 
password  will  be  changed. 

If  you  do  not  have  a  PIN,  enter  the  number  currently 
showing  on  the  SecurlD  card.  You  will  then  be  prompted 
to  create  a  PIN.  From  this  point  on,  you  must  enter  this 
PIN  into  the  SecurlD  card  to  generate  a  SecurlD  passcode 
Upon  creating  the  PIN,  your  Kerberos  password  will  be 
changed. 

If  you  have  any  questions  or  experience  any  problems 
changing  your  Kerberos  password,  please  contact  NAVO 
MSRC  User  Support  at  1-800-993-7677  or  via  E-mail  to 
msrchelp@navo.hpc.mil.  We  will  be  happy  to  assist  you  in 
this  process. 
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are  tested  in  parallel.  Such  a  parallel  evaluation  is  often 
inexpensive  due  to  the  hardware  independence  of  open 
source  code  and  minimal  time  invested  by  in-house  staff. 
With  such  a  small  investment,  the  trial  is  low-risk  and,  if 
the  tests  are  unsuccessful,  the  test  bed  can  be  discarded, 
and  any  hardware  used  in  the  testing  process  can  be 
integrated  into  the  current  infrastructure.  Often,  these 
evaluations  are  successful,  and  the  organization  can 
incorporate  the  open  source  technology  into  their  operations 
with  confidence  and  some  amount  of  experience  gleaned 
from  the  evaluation. 

A  Timely  Test  Bed 

In  1998,  the  VADIC  team  was  approached  by  the  Naval 
Oceanographic  Office  (NAVOCEANO)  and  NASA  Goddard 
on  behalf  of  the  Smithsonian  Institution  to  establish  an 
automated  SST  visualization  for  the  Smithsonian's  “Earth 
Today”  exhibit.  The  project  was  an  impressive  success 
story  for  the  NAVO  MSRC,  and  the  exhibit  won  the 
Smithsonian's  "Best  Virtual  Exhibition"  award  in  1999. 


Data  for  the  Smithsonian  and,  in  fact,  all  SST  images  are 
acquired  by  satellite  and  processed  by  NAVOCEANO. 
When  completed,  a  single  SST  image  displays  the  surface 
temperature  of  all  water  on  Earth.  (See  Figure  1,  page  13.) 
A  second  image  is  also  produced,  known  as  SST  Anomaly, 
which  illustrates  deviation  in  temperature  from  climatological 
averages.  SST  Anomaly  images  are  often  used  to  show  the 
well-known  “El  Nino”  phenomenon.  (See  Figure  2,  page  13.) 
For  over  6  years,  the  automated  process  for  the  Smithsonian 
images  ran  every  day  without  alteration.  In  late  2003, 
NASA  Goddard  again  contacted  the  VADIC  staff  to  request 
an  update  to  the  SST  visualization  application.  NASA 
Goddard  wanted  to  provide  newer  high-resolution  imagery 
of  the  landmasses  used  in  the  SST  visualizations  supplied 
to  the  Smithsonian.  This  request  coincided  with  the  VADIC 
staff's  research  into  replacing  the  legacy  system  with  an 
open  source-based  approach.  With  this  NASA  Goddard 
request,  it  seemed  the  ideal  time  to  update  the  entire  SST 
procedure  and  transition  to  a  new  system.  However,  there 
were  obstacles  to  overcome. 


SGI  Oivyx  2 
Opersling  Systenrn; 
SGMRIX 
Proce^sDre: 

B  MIPS 
al  2m  MHi 


Figure  3.  A  comparison  of  the 
original  and  new  systems  used 
for  SST  visualization. 
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The  Open  Source  Transition 

The  first  transition  challenge  would  be  to  determine  that 
replacing  the  original  eight-processor  Silicon  Graphics,  Inc. 
(SGI)  Onyx  2  with  an  off-the-shelf  or  commodity 
workstation  would  not  hinder  the  production  of  SST 
visualizations.  Although  SGI  is  still  one  of  the  leaders  in  the 
visualization  hardware  market,  this  specialized  hardware  is 
not  required  for  every  visualization  application.  The  best 
solution  for  the  SST  application  was  an  Intel-based 
machine  running  Red  Hat  Linux.  While  the  current  Linux 
systems  in  use  at  the  VADIC  have  a  single  processor,  this 
processor  is  significantly  faster  than  the  VADIC  SGI 
processors.  Another  major 
advantage  was  that  switching  to 
commodity  hardware  with  an 
open  source  operating  system 
reduces  maintenance  and 
operating  costs  by  a  factor  of  10. 
(See  Figure  3.) 

The  next  challenge  was  to 
transition  the  SST  visualization 
application  to  the  Linux  system. 
The  original  application  utilized 
Maya®,  high-end  rendering  and 
animation  software,  for  a  significant 
portion  of  the  process. 
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Origirval  SST  VisualSzaiicHi  Process 
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Figure  4.  (Left)  The  original  SST 
visualization  process  first  uses  a 
custom  data  converter  written  in 
C++  to  convert  raw  temperature 
data  to  a  256-level  grayscale 
image.  The  lowest  temperatures 
are  represented  by  darker  grays, 
and  the  higher  temperatures  by 
brighter  whites.  A  similar 
representation  is  used  for  below- 
average  and  above-average 
anomaly  values.  Maya®  can  import 
such  a  grayscale  image,  where 
each  grayscale  value  can  be 
mapped  to  a  color.  Since  no  actual 
temperature  values  correspond  to 
land,  it  is  left  black.  Mayaf§> 
performs  a  procedure  known  as 
compositing,  which  overlays 
NASA's  land  imagery,  but  only  in 
the  black  areas.  An  example  of  the 
resultant  image  is  shown  after 
each  step  of  the  process. 


Figure  5.  (Right)  The  new  SST 
visualization  process  utilizes  an 
updated  data  converter  that  also 
performs  color  mapping  and  uses 
ImageMagick  to  composite  the 
land  imagery.  An  example  of  the 
resultant  image  is  shown  after 
each  step  of  the  process. 
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?  composite  -compose  minus  sst_original.tif  land_only.tif  land_removed.tif 
?  composite  -compose  multiply  nasa_imagery.tif  land_only.tif  lancl_imagery.tif 
?  composite  -compose  plus  lancl_imagery.tif  lancl_removed.tif  final_image.tif 

Figure  6.  An  example  of  ImageMagick  commands  used  to  composite  the  original  SST  image  with  the  NASA 
imagery  to  produce  the  final  image.  These  three  commands  replace  Maya®  for  the  entire  compositing  process. 


One  option  was  to  simply  buy  or  transfer  licenses  to 
Linux.  This  would  produce  a  fast,  painless  transition. 
However,  Maya's  capabilities  far  exceed  the  requirements 
for  the  SST  visualization  application.  Since  the  application 
required  modification  to  accommodate  NASA  Goddard's 
new  land  imagery  request,  there  was  another  alternative. 
The  application  could  be  rewritten  to  use  open  source 
software  tools  rather  than  Maya®.  This  required  revising 
the  color  mapping  and  compositing  steps  in  the 
visualization  process.  First,  the  raw-data  converter  was 
redesigned  to  act  as  the  new  color-mapping  system.  This 
was  a  simple  modification  to  the  source  code:  rather  than 
translating  the  data  into  gray  values  from  0  to  256,  and 
then  into  color  values,  VADIC  programmers  directly 
mapped  the  raw  data  to  the  color  values.  Replacing  the 
proprietary  SGI  MIPSpro  compiler  was  an  essential  step  in 
this  transition.  The  data  conversion  code  was  ported  to  the 
GNU  C+  +  compiler  on  Linux,  and  then  the  color  mapping 
modifications  were  added. 


The  heart  of  the  application  transition,  however,  was  the 
switch  from  Maya®  to  ImageMagick  to  composite  the  land 
imagery.  ImageMagick  is  an  open  source  collection  of 
image  manipulation  tools  that,  if  desired,  can  be  grouped 
together  into  scripts  for  automated  processing.  VADIC 
programmers  were  able  to  combine  a  set  of  ImageMagick 
commands  that  performed  the  exact  same  compositing 
routine  as  Maya®.  Figure  4  depicts  the  old  SST  image 
process,  while  Figure  5  depicts  the  new  process,  utilizing 
the  new  converter  and  ImageMagick.  Figure  6  is  an 
example  of  the  ImageMagick  compositing  commands. 
Since  ImageMagick  is  an  open  source  software,  it  was  a 
very  attractive  alternative  to  Maya®.  While  it  cannot 
replace  Maya®  for  every  project,  ImageMagick  eliminated 
the  software  licensing  issues  and  replaced  the  image 
generation  functionality  for  this  project.  With  the  migration 
now  completed,  the  result  is  a  nearly  equivalent  system  in 
which  the  hardware  is  a  fraction  of  the  cost,  the  operating 
system  is  inexpensive,  and  the  tools  are  free. 
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